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ABSTRACT 

A 1987 project to develop materials for 
computer-assisted instruction in Thai and Indonesian and to train 
teachers in their use is described. Three different sets of Thai 
materials based on authentic materials were developed and field 
tested* One concerned the layout and function of a Thai monastery 
compound, another was an advanced exercise in Thai political science, 
and the third was an exercise in Indonesian-to-English translation 
from taped speech, a format selected from three that were field 
tested. The Indonesian test development and teacher training workshop 
occurred during the 1987 Southeast Asian Studies Summer Institute 
(SEASSI) , The final project report gives an overview of the project 
and its results and contains more detailed appendixes. The first 
outlines the formative evaluation of three types of Indonesian tests 
(a vocabulary drill, interactive exercise in translation from taped 
speech, and interpretation of a spoken narrative) and describes the 
SEASSI teachers* workshop* The second appendix contains a dot-^matrix 
printout of artist's drawings used in interactive lessons on the Thai 
temple complex. Appendix 3 contains a summary of the listening 
proficiency examination's development ^ reprinted from a journal 
article* The fourth appendix is a reprint of a book chapter 
describing the Northern Illinois University's Foreign Language 
Instruction Station (FLIS) instructional computer network, (MSE) 
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The goal of the project was two-fold: 



e Points ol view or opinions stated mthisdocu- 
n\en\ do not necessarily represent official 
OERt position or policy 



1. To create a series of advanced CAI lessons in Thai and Indonesian. 

2. To conduct a 5-day teacher-training workshop of SEASSI to introduce the new 
technology and materials. 

The project materials and various lesson designs were field tested and, in some 
cases, upgraded, in trials by students and SEASSI instructors. An outside evaluation was 
conducted by Prof. Peansiri Vongvipanond, Chair of the Department of Linguistics, 
Chulalongkorn University, Bangkok, Thailand. 

2. Development of Thai CAI materials. 

Three different sets of Thai CAI materials were developed and later improved after 
field testing with a limited number of students. They were all based on "authentic" 
materials, i.e., materials written in Thailand for Thais. 
A. "Our Temple" 



Description, layout, and function of a Thai monastery compound. This set was 
based on an illustrated book of that title and was chosen because of the central 
importance of Buddhism and the rituals that take place in different areas of the 
monastery. The original lesson design included oral presentation of a long stretch 
of text followed by a series of aural questions based on the text. The oral 
presentation was preceded by a listing of as many as 40 vocabulary items in Thai 
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orthography and English definitions which were also pronounced item-by-item via 
Instavox recordings. The student could not read the text on-screen, but rather 
listened to it in keeping with the premise of the comprehension approach to foreign 
language acquisition. In place of on-screen text, the student saw a picture of a 
temple structure (e.g., the ordination chapel) or activity (e.g., monks eating their 
pre-noon meai) while listening to explanatory or descriptive information. A series 
of true-false, yes-no, short answer questions followed. Statistics of the students 
right-wrong responses were reported at the end of each lesson/session. 

A graduate student of advanced Thai was asked to use the first series of 
lessons. He found that the chunks of information he heard were too long or too much 
to remembe*- by the time the questions were presented. On the positive side, he liked 
the lesson-opening presentation of the vocabulary and ability to access it later on 
demand. This series then was redone in smaller immediately interactive chunks, i.e., 
2-3 utterances followed by 1-2 questions. Then, after the questioning session, the 
entire stretch of utterances was played again for listening only - no follow-up 
questions. This final design has proven highly satisfactory among students (and 
SEASSI instructors) who have used it. A printed version of the text and glossary has 
since been prepared in xeroxed form for home study. It was used in the SEASSI 1987 
Thai class and *s now used by students at NIU and Berkeley. 

B. Advanced CAI Exercise in Political Science 

Because a good number of our advanced students major in political science and 
often do field research on the bureaucracy, it was decided to use official documents 
written in "bureaucratese." We chose an authentic Thai document on official 
government policy concerning development. It was signed by the then Prime Minister 
of Thailand and published in the National Gazette . Unlike the materials developed on 
the Thai temple complex, these had no pictures. The student was given intensive 
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practice in vocabulary in context provided by audio and video. Much of the formal 
lexicon used in government documents are Pali-Sanskrit in organ and difficult to 
pronounce, hence recall. The graduate students who used this set of materials found 
the ability to have difficult words repeated over and over at their simple pushing of 
a repeat key (R) a great learning assist. The quiz at the end of each section 
providing them immediate feedback was another positive feature. The fact that they 
were dealing with authentic government documents provided both an enticement and a 
challenge. 

Lesson Types in Indonesian and Their Forrrflative Evaluation 

Advanced and intermediate students in Indonesian (SEASSI 1987) volunteered to 
use and evaluate three lesson types: 

1. Vocabulary drill: 15 items to be learned. 

2. Interactive stoiy: brief story to be translated into English. 

3. Hyperspeech story: student enlists dictionary, grammar, cultural aids in 
understanding a chunk of spoken narrative. 

Results showed that students found type 1 ineffective. Types 2 and 3 were both well- 
received, but the hypertext format proved to be a bii more difficult for a few 
students, largely because of the increased complexity of the system, which will be 
streamlined in on-going projects. 

Pre- and post-tests of comprehension were given for each lesson type. Testing 
was conducted in English so as to reduce likely errors in the second language because 
active productive competency in demonstrating bare comprehension would be lower and 
cruder than simply stating in English what was understood (or not). Students found 
that pre- and post-testing actually helped them to focus their learning. Facility in 
comprehension decreased as chunks played became larger - a function of linguistic 
complexity and short term memory limits. In general, translation into English was a 



very useful evaluation procedure. Paper and pencil were used. Likewise the 
availability of English translations during the lesson (learning to comprehend) was 
a positive factor. Short multiple choice questions embedded in lesson types 2 and 3 
were recorded on the computer and registered on-screen. Student feedback on this 
type of interactive questioning was positive. Close tests were employed, but did not 
work well. 

The details of this part of the project have been written up in Appendix I of 
this report. 

SEASSI Teacher-Training Workshop with FLIS 

Several weeks prior to the conclusion of the ten-week SEASSI, the supervisory 
teachers of most of the language classes met with Professor Hartmann to receive brief 
training and exposure to CAI using the NTU FLIS. In order to do something 
worthwhile, we developed a simple, proficiency exam of listening comprehension. The 
primary function of the exercise was to introduce language teachers to new technology 
in the hope of spreading its acceptability and use in the future. At the same time, 
we were also sincere in wanting to see if the FLIS would be a reliable instrument for 
testing listening proficiency. As rough means of testing the validation of the exams 
for each language, we asked the teachf^rs to rank their students from high to low, and 
we compared their assessment with the computer results. In general, there were 
strong positive correlations, and we feel that with further refinements we could use 
the test to great advantage. Several students commented that they liked the content 
and style (social situations) as a learning experience itself, testing aside. 



Conclusions and Future Directions 



In all, the project was highly successful in term."^ of developing new CAI materials for 
advanced students of Thai and Indonesian. Beyond simply developing new materials were the more 
interesting experiments in formative evaluation of lesson types by students and the development 
and use of a listening comprehension exams in the major languages of Southeast Asia. 

Hardly had the project come to a conclusion when we found that the computer equipment 
(Apple He) we were using had gone from state-of-the-art in 1983 to obsolete in 1987. We have 
since moved to an IBM-PC environment with advantages too numerous to mention. With new 
funding, we will move the lessons to the IBM system and greatly update software for preparing 
lessons in hypertext. 

Dissemination 

The results of our work have been published in the CQTSEAL BuUetin (1988), F oreign 
Language Annals (1987) and in a collection of articles in a volume entitled Modern Technology 
in Foreign Language Education . In adi -'u >n, the FLIS system has been demonstrated by the 
authors at The Foreign Service Institute, University of Illinois-Urbana, and University of 
Hawaii. 

Explanation of Appendices 

Appendix I: Describes in detail the formative evaluation of Indonesian lesson types and the 
SEASSI teachers workshop for developing a listening proficiency exam. 

Appendix II: Dot matrix printout of artist's drawings used in interactive lessons based on 
the Thai temple complex. 



Appendix III: Results of the listening Proficiency Exam developed by SEASSI teachers and 
published in the COTSEAL Bulletin. 



Appendix IV: Dissemination: Offprint of "FLIS: Random Access Audio and Innovative Lesson 

Types" published in Modern Technology in Foreign Language Education, Edited by 
Wm. Flint Smith. Lincolnwood, Illinois: National Textbook Company, 
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A Descr i p t i on o-f 
Development o-f and Formative EM^.lu5,tion Ac t i i t i es -for 
CAI Lessons tor Foreign L^^nguages 
Conducted During the 
19S7 Southeast Asian Studies Summer Institute 



comp lied by 
George Henry 
September 24, 19S7 



This paper describes activities conducted during the 
summer o-f 1987 inuolvinq the development and evaluation 0+ 
C omp u t e r -A i d e d Instruction < CA I ) lessons tor -foreign 
languages. The lessons were developed under a grant* from the 
U.S. Department o-f Education, with additional -funding and 
resources supplied by the Southeast Asia Studies Summer 
Institute and Northern Illinois University. 

PART I - Environment and Personnel 

A. FLIS: The CAI materials under evaluation were 
produced to run on the NIU FLIS (Foreign language Instruction 
Station) system. The FLIS system -features a set o-f networked 
computer stations, each o-f which has an Instavox Random 
Ac cess Au d i o unit assoc i a t e d w i t h it. Th e Inst au ox a 1 1 ows 
interactive recording and editing of audio se gme n t s 
<" speeches") during lesson preparation, and instant playback 
o-f any recorded speech at any time during the use of the 
lesson. In short, any lesson which features an audio 
component is not limited to a preplanned rigid sequence of 
sp e e c h e s , but may be designed to react to event s oc c u r ring 
during the lesson ( su c h as learner choice or the correctness 
or incorrectness of the 1 e a. r n e r ' s response t c« a q u e s t i o n ) . 

Further details about FLIS, its underlying philosophy, 
and the basics of the lesson types used can be found in 
"Computer-Controlled Random Access Audio in the Comprehension 
Approach to Second Language Learning" wrhich is attached to 
this paper as an appendix, 

G. STUDENTS: The lessons being evaluated were taken by a 
number of students from SEASSI - Southeast Asian Studies 
Summer Institute. SEASSI is run by a consortium of U.S. 
universities, and each summer brings together about 30 
teachers (mostly from Southeast Asia) and over a hundred 
students (mostly from the U.S.) to teach approximately a 
dozen Southeast Asian languages and the cultures of the area. 

The students are selected by the SEASSI board on the 
basis of ability J interest in a field relating to S.'outheast 
Asia, and projected ability to benefit from the instruction 
at SEASSI . Host are graduate students or advanced 
undergraduates, and most may be described as highly 
mot i vated . 
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Those who took the lessons we re yoi 1 un t e e rs from the 
intermediate n d advanced Indonesian c 1 a s se . It might h a u e 
been preferable to ask the language teachers to reouire these 
lessons as part o-f the SEASSI course, but this was not 
practical for 'various reasons. Those who volunteered might 
be imagined to be more highly motiuated than euen the aueraqe 
SEASSI student, and to be -Favorably inclined to CAI . 
H owe e r , as a n a. 1 y s i s of their r e a ■! t i on s will ^ h ow , t hi e r e w ere 
a -Few amon g them wh o were highly critical oi on e asp e c t or 
another oi the lessons, and most had at least a tew 
suggestions and criticisms to offer. 

One additional factor that should be noted is the great 
amou n t of wo r K and intense pressure that these s t u de n t s 
experienced. Language c 1 asse s we re held 4 hours a day , f i u e 
days a week for 10 weeks, with daily homework assignments. A 
typical class had 4 to 5 students. Most students took 
additional coursework such as anthropology classes. They 
came to take the CAI lessonsj then, after 4 to 6 hours of 
intense instruction and recitation, usually on a day where 
the t e rti p e r a t u r e wa ^ ab o u e 9 0 . These w ere not optimum 
conditions for learning, and might well haMe reduced any 
possible predilection toward high motivation and Interest. 
However, since participation was 'v-oluntary, the only 
rrio t i V a t i on involved was intrinsic: no extrinsic m o t i v a t i on 
was offered. It should be noted that we owe t h e m sp e c i a 1 
thanks for their interest and efforts under difficult 
c i r c umst ances . 

Because of the fact that participation was voluntary, 
the number of s.tudents taking the lessons was rather small, 
ranging f r om 9 down t o 3 f or p ar t i c u 1 ar 1 esso n s . Th u s 
c on c 1 u s i ons for individual 1 e sson s , i n somie c ase s , miu s t be 
tentative. However, 9 different lessons were taken for a 
total of 50 s t u de n t - 1 essons , so conclusions for the system d^.s 
a wh o 1 e c an be r e 1 a t i v e 1 y f i rrri . 

C . STAFF : The activities we re directed by 6 . Hen y and 
J. Hartrtiann. Most of the work of lesson development and 
administration of the lessons on a daily basis was done by 
student workers, trained and rrianaged by Henry and Hartmann. 
If the FLIS system is to become an operational success, it is 
inriportant that people can be trained to create lessons for it 
and to run it. A later section will deal with these 
quest i ons . 



Par t II - Goal s 

The FLIS system has as one of its primary goals the 
presentation of instruction whi i c h all ows learners to p r a c t i c it 
and i rtf p r o V e their listening c om p r e h e n s i o n of f or e i gn 
languages, and to build their passive vocabulary, in an 
effective, interesting, and rewarding way. 

In connection with this, a specific goal of our 



i J 



BEST COPY AVAILABLE 



activities this summe r was t o de'j i se i.gork a.bl •=• test} n q 
procedures to ^-e term 1 n e i -F , -For a given lesson, the student 
had 

a) understood the special uocabul ary pre-Lented in the 
1 8 s s o n 

b) understood representative speeches well enough to 
explain or translate them 

c ) had under s t ood the c on tent of t he s t or y we 11 en ou qh 
to summarize the story and 

d> had understood the story well enough to answer 
q u e s. t i o n s i n v o 1 'v'i n g i n -F e r- e n c e s- . 

It should be noted that this is a dit+icult problem, 
since comprehension is essentially an internal mental state, 
We rriust ask the student to per -Form some action in order to 
assess the degree of comprehension attained, but we friust try 
to ensure that a wrong answer (action) is not due to the 
inability to per-Forrri the action when comprehension was, in 
•fact, attained. For this reason, rriost questions are posed in 
English and most responses are given in English, 

In addition, we wanted to determine which C i t any) o-F 
the lessons were perceived by students as interesting, 
e-F-fective, and use-Ful. A questionnaire was given to each 
student after each lesson in order to obtain reactions, In 
addition, ■Free-*form comments were requested on these -Forms, 
and verbal comments were also sought. In some cases, 
suggestions for improvement were incorporated in lessons 
under de v e 1 o p m e n t , 

Par t III - Lesson Type Corripar i son 

For the -first three weeks of the study, each student was 
to take 5 lessons: one o-f each of the nr|a j or types developed 
so -Far. More extensive descriptions o-F these types can be 
-Found in the appendix^ the FLA paper), but briefly, they are: 

a) Vocabulary drill: 15 i terris are to be drilled until 
1 e a r- n e d . A p r e - 1 e s- 1 is. f i r- s. t a. dm i n i s t e r e d in w hi i c h each item 
is played (spoken) once: the student is to write the word's 
English meaning. Then the drill lesson begins: each word is 
introduced with its English meaning displayed on the screen; 
then the items are practiced randomly until the student feels 
he has learned the words and wants to stop. Then a post -test 
identical to the pi r e - 1 e s. t is given. 

b> Interactive story: A pre-test is first administered 
in which sample vocabulary (both word and sentence level) is 
played (spoken)" the student is to write the English meaning. 
Then the story begins: speeches and pictures present a story 
w»hich can be controlled at certain points by the student. 
Choices iTiay be made as to where to go, what to buy, what 
price to offer, and so on* The student is encouraged to 
repeat the story, making different choices each time, in 
order to experience repetition with variations. Sortie 
erribedded questions are co^t tained in the lesson, A post- test 
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idf-ntical to the pre -test is then given. 



c ) H >' p e r- s.p e e c hi : A p r e - 1 e s. t is- t i n s t a dnri I n i s t e r- e d in 
u>h i c h s 5^rri pie v o c a bu 1 an >' C bo t hi vm o n d and sentence level > i s 
played (spoken) i the student is to ignite the Engl i sh meaning. 
Then the storx on passage begins: each sentence on two 
("chunk") is pnesented sepanatelXj with various aids 
available to the student to "help him understand the passage. 
These aids are: t r e^nsl a t i on , gnamman , on cultunal notes 
available as "hints"; a slower re-recording of the speech, at 
well less, than native speaker speed the display ot the text 
of the speech on the screen as it i s be i nq spoken; and 
finally, it all els.e fails, an English translation ot the 
"chunk". The student is tree to make use ot any on all ot 
these aids, although some nestriction is made as to the 
order i n wh i c h they can be accessed. For e x am p 1 e , the 
student must work through some at the other aids before the 
English translation is available. No questions are asked 
during the lesson. After the lesson, a post-test identical 
to the p r- e - 1 e s t is g i y e n . 

Summary of res.-ults 

After taking all 3 lessons, students were asked to rank 
themi specifically they were asked 'Which lesson type did you 
prefer?"' The hyper speech lesson was preferred by 4 of the 
s i X wh o an sw e r e d t e question, t hr e interactive e- t o r y b:>' 2 o f 
the 4 J and the drill (not surprisingly) by none. The 
interactive stony lesson was second for- 4 out of the 6. The 
drill lesson was ranked 3rd by two; the other 4 did not rank 
it. 

Af ten each of the 1 esson s , each student was ask e d to 
fill out a " CAI Lesson Eval uat 1 on Form" ( a copy of wh i ch i s 
attached). Considering the questions that nelate 
spec i f i r.=i.l 1 y to lesson type, the following conclusions can b- 
stated! 

- students n a t h e n disliked the v oc abu 1 an y dr ill 1 e sson , 
and found it boring, although it was at the right level for 
them and was not too long on average. They considered it to 
be rather ineffective. They would not recommend lessons of 
this type to others. Comments tended to complain that 
learning words in isolation was not helpful, and that the 
words would not stick with them. One student noted later 
that she encountered 4 of the wonds the following week and 
could not remember their meaning (although she did apparently 
remember that she had seen them). 

- students liked the interactive story and the hyper- 
speech , an d f ou n d them effective, interesting, and clear, if 
some wh at challenging. Audio q u a 1 i t y was a problem to some , 
(The m=^terial for the hyper speech lesson was taken from a 
cassette tape of marginal quality). Thene was a spread of 
opinion on the difficulty of the lesson, possibly reflecting 
the difference of abilities of the students. Overall, there 
do not seem to be any serious problems. Few comments i.-Jere 
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made regarding the I n t er ac t ; Me story lesson: students st-erried 
well satis-fied with it. A number o-f suggestions were made to 
i mp r ov e the h y p e r sp e e c h 1 e sso n s , some o t wh i c h we re adopted 
in subsequent lessons, a.nd others either rejected or rem^Jn 
to be imp 1 emen ted . 

- in most cases, use o-f the system did not pose a 
problem. That is, instructions, choices, questions, and 
options in lessons did not cause any systematic problem. 
There were a. tew isolated problems caused by misreading 
directions, pressing the wrong key, and in a few cases, 
problems in the lessons themselves (which were fixed as they 
were identified). Since this was the first heauy test of the 
system, it was gratifying to see that in no case did the 
underlying s. o 1 1 w a r- e fail. 

- the pre- and post-tests all showed si gn i 1^ i cant short 
term learning o-f vocabulary, except -for a couple o-f students 
who claimed that the hyperspeech lesson was much too hard. 
i/Je are not satis-fied with* this method for testing, and -feel 
that no claims about the comparative value of one lesson type 
over another can be made on this basis. Student c omme n t s 
that long-term retention is not guaranteed (or even probable) 
may be valid. Since the tests did not ask questions about 
the meaning of the stories- as a whole » it is difficult to 
assess compr ehensi on . 

Fart lU - Hyperspeech Lesson Development 

A. Less^ii Template Development 

Comparison o-f the three lesson types indicated that the 
interactive story and the hyperspeech lesson types held 
c on s i d 9 r a b 1 e p r om i s e . The vocabulary drill t y p e , if useful 
at all, would need considerable rethinking, We decided to 
concentrate on the hyperspeech type, -for one reason: 
simplicity of lesson creation. The Interactive story lessons 
require considerable creativity and ingenuity in their design 
and construction. To create, in effect, a dramatic story 
with multiple realistic turns of plot is not easy; to create 
a set o-f such lessons is. quite a trick. (However, control o-f 
vocabulary is easier, since the stories are written, rather 
than "found".) hyperspeech lessons, on the other hand, are 
quite mechanical to produce, once the raw material -for the 
lesson hi a s been identified- The 1 e s son author h as to 1 ) 
choO'=e , transcribe and translate the passage, 2) break the 
passage down into appropr i a te 1 y sized •'chunks'' , 3) i den t i -f y 
di-f-ficult vocabulary or grammatical points, and perhaps some 
relevant cultural notes for hints, 4) design the pictures 
used to illustrate the passage, and finally 5) record thc- 
speeches, hints, si ow speeches, and translations. The 
e 1 abor a t i on s to the ''bare'' h >'p e r sp e e c h lesson which we 
ultimately decided upon are also straightforward and 
me c h an i c a 1 , 

Over the course o-f about 6 lessons, the following 
enhancements to the basic hyperspeech -framework were made: 
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- mor e el5.borate i n t roduc torv rriAterial , speci-fyinq the 
source o-F the p8.ss^ye and qiuing a brief description ot its 
c o n tent , subject matter, and of the testing met h od t o be 
used. This ms^terial may function, in part, as an Advance 
Organizer 'vAusubel). Some students commented that Knowing 
what kind o-F test would be giuen helped them -focus on the 
materia] . 

- addition of a number o-F embedded questions between 
some o-f the language "'chunks''. The FLIS system allows both 
multiple choice and short answer questions, but we used 
exclusively multiple choice questions. If the student missed 
the question, the program would replay the speech that 
contained the an swe r , and the student could then t r x the 
question again. Again, students ri o t e d that hi a v i n g questions 
interspersed amon g the speeches i n t r odu c e d variety wh i c h kept 
them from automatically drifting through the material. Some 
noted (again) that knowing they would have to answer 
questions helped them focus and concentrate. 

- the addition. of multiple named hints. In the first 
hyperspeech lesson, there were only 2 hints per -'chunk'. 
The student was instructed to press a key to hear them, 
without knowing what he would hear. Understandably, this 
proved -frustrating i -F he wanted to hear, about X, but 
instead were told abo u t Y wh i c h h e ma >' w e 1 1 h a v e k n own ) . 
Ule were able to devise a way to provide as many a^. 7 hints, 
and to label them on the screen: 

1 = Siska 2 =. grammar o-F "di- -kan" 3 = cultural note 

On e suggestion that w ou Id require a. c h an g e to the 
so-Ftware would be to allow the playing o-F a hint without the 
subsequent repetition o-f the speech for the -'chunk'. Uh i 1 e 
repetition is doubtless valuable, it becorries an irritation 
at some point. Better to leave it under learner control. 

- a complete, continuous playing o-f the passage be -Fore 
the breakdown into 'chunks'; that is, be-Fore the hyperspeech 
component o-F the lesson. 

Thus, the -Final versioi. --i-F these lessons begins (a-Fter 
the lesson introduction) with an uninterrupted playing o-F th 
passage. The student listens to the whole thing without 
pause, to pick out whatever he can, unaided. Euen i -f large 
parts are missed, this gives the student a preview, and help 
him with the overall context and gives clues o-f what to 
listen -For later. 

- addition of* a p' r e - 1 e s t '^ a -F t e r the continuous playing 
o-F the passage) and an identical post-test (after study of 
the h/perspeech potion of the lesson). 

Typically this test asks for translations of a few of 
the vocabulary items in the passage, for a translation of a 
few of the sentences in the p assage , and f or an s im e r s (in 



Eriylish) to a. tew questions about the ^torv (either tactual 
and/or i nt eren t i a) ) . (See also notes on Testing in the ne>it 
section.) At an y time, the student can r e p 1 ay the ujh o 1 e 
uninterrupted passage, Care is taken to in-form the student 
at each stage ot the lesson what to expect, what is required, 
an d wh at the au a i 1 ab 1 e op t i on s ar e . A 1 t h ou gh i-Me we r e ab 1 e to 
c omp 1 e t e on 1 y one sue h "'enhanced' 1 e sson j i n t or ma ] c onrinrie n t s 
by departing students lead us to belie's-»e that this is a 
su c c e ss-f u 1 lesson pattern. 

Lesson Spec i f i cs 

A brief description ot the various ( hyperspeech ) lessons 
used toll ows : 

Hunting: a dialogue between two Indonesictn comics about 
how they hunted wild game in the old days. Separ a t e"^ p r e- and 
post-test. 1 hint per trame (= 2 per set). No enribedded 
quest i ons . 

Selling cloth; another conrnc dialogue relying on a pun. 
Students are to write an English sunrimary. i hint per frame. 
No embedded questions. 

Tua; a part ot* a, rriodern Indonesian short story; sonrie 
wihat abstract and surrealistic. Includes enribedded questions. 
Mu 1 t i p 1 e hints per trame . Test is to wr i te Engl i sh summary 
and do a aural cloze test, 

Lagu : two Indonesian pop songs. Not a true hyperspeech 
lesson, but sinriilar. No embedded questions or hints. Urite 
English translation ot songs as test. 

B a k s o : A s t o r- y f r- ont a n e vj s p a p e r ab ciu t a t o o d •=■ t a 1 1 whi o s. e 
popularity was due to flavoring by "ladies' unde rvjear " , No 
enribedded questions; miu 1 t i p 1 e hints per franrie. lOritten 
English sumima.ry and both a written And aural cloze test, 

Masai ah PenduduK; a explanation of Indonesian's 
population probl erri and somie atterripted solutions. Has 
rriu 1 1 1 p 1 e hints per frame and emibedded questions. Ur it ten 
English sum»mary and both a written and aural cloze test. 

Si ska: A condensation of a modern Indonesian short 
story about a young girl's feelings of restrictions and 
alienation. Has rriu i t i p 1 e hints per franrie and embedded 
questions. Uses story - pretest - lesson ~ post-test pattern 
as described below. 

Part U - Testing Method Development 

The difficulty of nrieasuring foreign lan'-yuage 
conrtprehens i on without requiring foreign language production 
has already -^een nrien t i oned . One way to reduce the difficulty 
is to ask questions and elicit answers in the learner's 
native language <English, in this case). Foreign language 
t e ac hi e r s generally do not encourage translation < or s i m i 1 a r ) 
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activities on the grounds that it inter -feres i.-.iith le^rninQ; 
that the student should learn to think directly in the target 
language. However, i +' it is important that we be able to 
gauge understanding, we think that translation type exercises 
are required <at least uhen restricted to written responses). 
li questions are posed in the -foreign language, one can never 
be sure, in the case of a. tgrong ansujer, if the failure was a 
lack of understanding of the target material or the question 
i t se 1 f . If the response is required to be in the foreign 
language, and is wrong, one cannot know if comprehension or 
production is at fault; that is, the student nriav have 
understood the passage, but not have been able to produce the 
correct answer. It is well known that learners-" passive 
vocabulary and comprehension skills are nriore advanced than 
their production skills. In short, if we want to measure 
c onrip r e hi e n s i on without involving other foreign languages 
skills as well, we nriust do the testing primarily in English, 
or ensure that the difficulty of questions and answers in the 
foreign language are carefully controlled and are well below 
the level of corriprehens i on being tested. "^he level of such 
quest i on s nri a y well be set on 1 >' by trial a n d e r r or , a 
difficult and e x p e n •=. i v e task. 

Another difficult problem in the assessnrient of student=-' 
conripr ehen s i on of these lessons is the provision in the 
lessons for the playing of an English translation of each 
"chunk" . Transl at i ons are prov i ded for tujo reasons: f i rst , 
nrian y students use t h e nri to check t he ac c u r ac y of t hi e i r- 
understanding. They find the confirmation of their 
understanding reassuring, and the instant correction of an >- 
nrii sunderstandi ngs valuable. Second, in the event that the 
PTieaning of the "chunk" remains obscure after all the other 
aids have been used, the translation provides an u 1 t i nri a t e 
rerriedy, allowing the student to understand and proceed. L-Je 
do force the student to access other aids before the 
translation, but it is always ultimately available. 

The difficulty is that when we ask the student to explain the 
story, or infer something from it, we cannot say whether the 
(correct) explanation is due to the student's understanding 
of the foreign language or, in whole or in part, of the 
English. Somie students commiented that the requirement to 
sufTimarize the passage was useless for just this reason. 
Others, however, conrimented that knowing they would have to 
suminriarize the story helped them concentrate. For less well- 
motivated students this requirement might help ensure cha.J: 
they pay attention to the whole passage. 

This difficulty is not so apparent if we ask for a 
translation of a woi-dj phrase, or sentence from the passage, 

In s u mnri ar- y , we feel t hi a t t hi e English t r a n s 1 a. t i on shi o u 1 d 
stay. Granted that a student could use only the translations 
to answer the content questions, we think that if a nunriber of 
questions based on the foreign language are also included, so 
that the content questions are not the only measure of 
understanding, a decent nrieasure of the student's 
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comprehension c^n be obtained. 



I n s om e 1 e ^-s ci n =- , vj e u sed c cmri b i ri =<. t i o ri ■:■ + =• e v e r a 1 
techniques. The -foil ow i n g is -=«. s ummar y oi testing techniques 
tried with c^n ^^ssessme n t of results. 

- separate (identical) pre- and post-tests utilizing 
word/phrase translations. The word or phrase to be 
translated was played oncej the student was instructed to 
write the translation on paper. This tc-chnique seems well- 
suited to measure 'Jocabulary acquisition and to sample 
understanding at the sentence level. Usually the student 
shovjs considerable gain after the lesson. Possible -flaws are 
1> the gain miav be only temiporary; a delayed test on these 

i temis given later would be helpful and 2) the pre- test nriay 
help the students -focus on the tested items during the lesson 
in preparation fcr- the post-test. This latter -f 1 avj mi^-// 
encourage students to ignore non-tested items. 

- English sumimiary. As mentioned above, this technique 
su -f -f e r s -f r om the p r o b' 1 e mi that t hi e supplied English 
translations in the lessons may supply the knowledge needed 
to write a translation. In addition, somie students -find it 
useless and irritating -For this very reason. On the other 
hand, this technique does help others attend to the lesson. 
In add i t 1 on 5 the mi i s t a k e s mia de i n t h e sumimiar y are a valuable 
indication oi what was mi i su n d e r s t oo d . Bee au s e oi the 
advantages, we think that this technique could be used 
(perhaps not -for every lesson) \i somie way could be found to 
reduce the students'' negative feelings. 

- Embedded questions. Short answer or multiple choice 
questions c a ri be placed at intervals in the lesson. The 
correctness of answers to these questions can be recorded by 
the FLIS systemi and printed later. lOe chose to present 
multiple choice questions, displayed on the screen i .e. the 
student read rather than heard themi) in the foreign language. 
A conscious effort was miade to use only simple vocabulary 
plus vocabulary from the passage In the questions and 

a n sw e r s . Student c cimimi e n t s ori these question s Iaj e r e pi o s i t i v e , 
citing the attention-focusing effect and feedback that they 
provide. The computer record of these questions indicated 
that only one student on one lesson <out of 11 student--on- 
lessons) miissed any of the questions. 

A possibly useful source of information on individual 
learning styles during these lessons was noticed while 
examilning the printed output of these- questions. Since the 
computer records each frame tried (as well as numiber of 
attemipts at a question and the final correctness of the 
response), and prints frames not tried as dashes, V4e can s.ee 
fairly easily which frames in the lesson were tried- It Is 
apparent at a glance, for examiple, that somie students rarely 
or never accessed the help f ramies (slow speech or 
text/translation) during certain lessons; others miade. heav i er 
but not constant use of themi, and a few methodically tried 
every available f ramie. One student tried every frame of a 
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lesson until he became irustr c«. t^-d and quit it entirel>- be-fore 
•finishing. (His frustration im as. r e pi or t e d to me or a 1 1 y . ) 

- Cloze tests. Did n o t wor k well. We t r i e d t w o t /'[n e s : 
uisual (receding) and aural. A sample ot ti-Jo to three 
sentences -f r cm the lesson was either d i s pi i a / e d on the scree n 
or read aloud. Euerv se'jenth word was replaced by a 
(numbered) blank or a spoken number (in English). Studenta- 
•found these di-f-ficult and frustrating; the aural cloze was 
almost impossible, Atter trials in 2 lessons, this technique 
was rejected, 

- Pre- and post-test in the lesson using uarious types 
o-f questions. This differs from the separate pre- and post- 
tests mentioned abo'je in two important respects. First, the 
pre -test is- presented after the student has heard the stor>' 
once, withojt pause, all the way through. Therefore it is not 
1 imiited to measuring what the student knew before the lesson, 
but can now ascertain both what the student knew before the 
lesson (vocabulary) and what he has understood without aid 
having heard the passage once- Second , since the student has 
heard the passage, we can now ask content questions as well 

as bare vocabulary and phrase translation questions. The 

difference betiMeen pre- and post -test then is a measure of 

what the student has learned from the lesson that he was not 

able to understand on his own. The two possible 

d i s- a d y a. n t a 9 e s. o f h a i n g identical p r- e - and pi o s t - t e s t s 

men t i on e d abou e still pertain, h owe v e r , 

The test questions given include several types: 

1. 'v'ocabulary wiords displayed on screen; stucent to 
wr i t e English t r an s 1 a t I on/ e x p 1 an a t i on 

2. Phr ase s/se n t e n c e s from the passage played -'one at a 
time); student to write translation of each 

3. Factual questi on s abou t the s t or y d i so 1 a ye d on screen 
in English; student to write the answer in English 

4 . Inferential questions abou t the s t or y d i sp 1 aye d on 
screen in En g 1 i sh \ student to wr i t e the an swe r in En g 1 i sh 

Note: 3 and 4 might be asked in the foreign language if 
the vocabulary was kept simiple. We have not tried this yet, 

These enhancements, then, fit together as follows: 

1 . Directions and I n t r odu c t i on 

2. Listen to the passage without pauses 

3. Pre- test; answer questions about the passage 

4. hyperspeech presentation of passage in "chunks" with 
aids; includes embedded questions; may he repeated at will 

5- Post -test: identical to pre- test 

Part V'l ~ Sumimary of Student Quest i onai res 

Following each lesson, each student was asked to fill 
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out A "CAi Lesson E^j.=i1 us.t i on Fornri" . The results were 
t&bulated «<.nd ex^^mined. A copy oi the -forfTi is supplied as 
an cvppendix. One gener^.l suggestion -For the -fornrr is th^.t an 
odd n u mbe n o-f gradations be used I n the +' u t u r e , so t h a. t 
students rri.=i>' rric^rk an exe.ct middle or neutral value. This 
section summarizes the results and adds comments: 

Section 1: agree - disagree \\ \ - 6 

1, I liked this lesson. 

Uocabulary drill: a^.'erage 4.42 = mild disagreement 
Interactive story: average 1.44 = strong "agreement 
Hyperspeech range: 1.33 - 3.67; average 2.40 = ok 

Hyp e r sp e e c h 1 e sson s that i/je re rated 1 e s 'likeable' 

- were relatively hard or 

- were subject to c on s i de r ab 1 e d i -F -f e r e n c e o-f opi n i on or 

- used the uni-formly unpopular cloze test or 

- were taken by -few (2 or 3) people, one o-f whom was P, 
a uniforiTily critical student who nonetheless insisted on 
taking all possible lessons and claiming t hi a t each was 

i mp OSS i b 1 y d i -f -f i c u 1 t . C H e was , h owe v e r , enthusiastic about a 
children's story starring a naughty monkey. This story wias 
not included specifically in this study.) Incident] y, there 
was no correspondingly positive student in the study. 

Note that '1 ike" may be taken to mean (in part or in 
wh die) c on t e n t J difficulty, testing method, or s orrie o t h e r 
factor. Future versions of the questional re might try to 
distinguish more clearly among these alternatives. 

5ee also Part II questions which rriight reflect on 
overal 1 '1 ikeableness' . 

2, Th i s 1 esson was at the r i gh t 1 eve 1 for me . 
Uocabul ary: average 2.17 = mild agreement 
Interactive story: average 2,39 = rri i 1 d agreement 
Hyperspeech range: 1.33 - 3,75; average 2.68 = ok 

This question does not distinguish 'too hard^ fronri -'too 
easy' in the case of disagreement. However, also see 
questions for Part II, which might be correlated with this, 

There were few strong disagreements with this statement. 

It is quite true that a couple of the 1 e sson s w ere in 
f ac t quite hard , 

3. 1 kept wishing the lesson would end. 

V'ocabulary: average 2.47 = mild agreement (range: 1 - 5> 
Interactive story: average 4.53 = mild d i sagr e errien t 
Hyperspeech: range 3.25 to 6\ average 4.75 = 
d i sagr eemen t 

The vocabu 1 any drill is fairly bor i ng 5 st ill, some 
students did not find it irripossible in this respect. 
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Most found the interactive story -fine; 2 out oi 9 -foun-j 
it too 1 ong . 

A couple oi the longer or harder hyperspeech lessons w^t 
■found to be too long by some students. This may also be in 
part due to the forced repetition of each speech a-fter a h i r. t 
is played: if all hints are played, one may be forced to 
listen to t hi e speech a =• m a n >' as 10 t i m e •=. » In general, 
however, the length of the lessons is not a problem. 

4. Use of the equipment was sometimes a problem. 

On all le-=.sons; range was 3.75 - 6.00; average 5.34 

In general, there i s n o sy s t errra tic p r- cib 1 em . S ome o f t e 
specific problems >Atere due to bugs in the lessons (wrong 
branch, etc.) which were quickly corrected. A few times, 
students hit the wrong key (i.e. Quit the lesson instead of 
request Translation). Audio quality (see 7 b e 1 ow > was 
sometimes a problem and may have affected answers to this 
quest i on . 

Given the fact -that students were given minimal 
introduction to the system, were presented with a different 
lesson type each week, and were generally hot and tired by 
th^ time they took the lesson, the lack of problems in thi-= 
area is very encouraging. 

5. There were time= when I wasn^'t sure what to do next. 
On all lessons: range was 4-6; average 5.04 

Aqa i n j n o sy s t ema t i c p r ob 1 em ; a i e w sp e c i f 1 c i so 1 a ted 
problems. Se v e r^a 1 we re related to t hi e tests ( wh i c h c h a n g e c 
i n f orma t a 1 mos t every 1 e sson ) . 

6. More lessons of this type would be useful, 
^-loc abu 1 ar y : average 4.57 = disagr--:-. 
Interactive story: average 1.67 = strong agreement 
Hyp e r sp e e c h : range 1.67 - 4 . 25 ; a v e rage 2.54 = agr e e 

The vocabulary lesson was not seen as useful by most 
students. 

The interactive story was seen as quite useful . More c 
these should be developed, we believe. 

H y p e r sp e e c hi v ar i e d consider ab 1 y . Tu a , i.m i t h t h ^ 1 owe s t 
(4.25) rating here was probably the hardest (and strangest). 
It is not clear why BaksOj with the second lowest (3.33) wa= 
not seen as more useful. All other hyperspeech lessons 
averaged less than 3 on this question. 

Note that ^of this type^ may be ambiguous: it could be 
taken as lesson type, or content, or- testing method to some 
degree . 

7 . Audio quality was s orri e t i me =. a p r o b 1 erri . 

On all lessons: range 3.50 - 6.00? average 4.81 = mild 
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Overall this i s n o t a se r i ou s p r ob 1 em , but in some 
1 e s s on s abo u t half of the students indicated p r ob 1 em s . S om e 
Q-f these were the lessons copied -from an already-' marginal 
cassette tape recordinq. Others seemed oK to me. Possibly 
these comments relate to individual passages, not the whole 
lesson. Generally, we must be care-ful about audio quality; 
if not attended to, it can bee ome a p r ob 1 em . L i e r e c or ding s 
are usually better than tapes; if tapes are to be used, the y 
must be of uery good audio quality. 

Also interesting is the fact that about half the 
students claimed no problem at all, even on the lower quality 
aud i o mater i al . 

8. Video quality was sometimes a problem. ^ 
On all 1 essons : range 4 . 75 - 6 

Not seen as a problerri. Of course^ the focus of the 
lessons is on audio; still the pictures contain contextual 
clues and least) should not interfere. See also question 

1 1 . 

9. I felt under too miuch pressure to get all the ansiMers 
r i qh t during the lesson. 

On all lessons; range 4.33 - 6 

In the context o-f the sumrrier. this did not seem to be a 
problem with these students. Different students, taking 
lessons for part of a grade may -feel different. 

10. I wou Id r e c omme n d 1 e ssons of this type to o then s 
interested in learning the language. 

Vocabulary: auerage 4.42 = not really. 
Interact i '.'e story: auerage 1,63 = strong yes* 
Hyperspeech: range 1 - 3.25; average 2.39 = yes. 

Again, no to vocabulary., yes to the story, and yes to 
hyperspeech J fjith certain of the hyperspeech lessons getting 
only mild recommendation. Also, again, 'type*' may be 
amb i guous . 

11. The pictures were helpful if present. 

All lessons: range 1.33 - 4.33; average 2.79 = mild yes. 

In general J there is mild agreement here. Some 
individual pictures in some lessons are probably a bit 
confusing and could stand s.ome improvement (e.g. unclear 
which picture is which character). Except for the 
vocabulary drill lesson, students had no lessons without 
pictures, and so no way to c omp an e pictures v s , no pictures. 

Section 2: 6 point scale between opposites: 6 ... 1 

1 . bor i n g . . . i n t e r e s t i n g 

Uocabulary drill: average 5.14 = pretty boring 
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1 n te r .=^c t i ^ ? story: 3.wer^g9 1.56 = pretty i n t erest i rig 
Hyp e r sp e (£• c h : range 1 . 33 - 3.33; average: 2.25 = ok 

Aga i n , am on g the h y p e r sp e e c h ] e sson s , the harder 1 e sson s 
and those with P rated around 3; the others between 1 and 2. 

2 . i n e t -f e c t i V e . . . e -f -f e c t i ■ j e 

Vocabulary drill: average 4.00 = somewhat i ne + -f ec t i ye 
Interactive s t o r- y : average 2.11 = ok 
Hyperspeech; range 1.33 - 4; average 2.53 

SifTjilar to 1 above. Students tend to reject materials 
which they view as i n e-f -f e c t i v e , so it Is good to see that 
their reaction to t h is q u e s t i on i s p os i t i 'v' e . 

3. con-fusing. . .clear- 

Uoc a bu 1 ary dr i 11 : average 1 .S3 = clear 
Interactive story: 1 . 44 = very clear 
Hyperspeech: range 1.67 - 4; average 2 . S3 = ok 

"Con-fusing" miay re-fer to directions, speech, story line, 
or- testing ma t e r i a 1 s . This q u e s t i ori needs to be clarified. 
Also, Tua.j <4.0D> is inherently somewhat con-fu-sing. 

4. too hard. . . too easy 

'v'oca-bu 1 ar y drill : average 3.2 = just about right 
Interactive story: average 5.00 = just about right 
Hyperspeech: range 3.75 - 4 . S3 ; average 4.32 = a bit 
too hard 

The h y p r sp e e c h lesson s w ere g e n e r a 11 >- c on s i de r e d j u t a 
bit on the ha.rd^side, which was the intent. The student is 
e X p« e c t e d to won k and dig a b it to u ri d e r s- 1 an d e v e r- y t h i ri q . 

5. too long... too -short 

'v'pcabul ary drill : average 4.20 = a bit too long 
Interactive story: average 3.11 = just right 
Hyper-speech: range: 5.33 - 4.75; average 4. OS = a bit 
too 1 ong 

Again, removing the requirement to 1 isten to each 
speech over again after each hint would help in this area, 
although a 4 is not a strong indication. 

6 . frustrating. . .satisfying 
Uocabulary drill : average 3.00 = middle 
Interactive story: 2.44 = rather satisfying 
Hyperspeech: range: 2.53 - 4; average 3,36 = middle 

For many of these lessons, the range of answers was 
large. Students disagree about this an-swer to a large 
extent. In each case it not dear wh ether the 
frustration may be due to difficulty of vocabulary, testing, 
audio quality, etc. It is not clear either what aspect seems 
satisfying. This question might be better omitted or changed 
to be more specific. 



1^ BEST COPY AVAILABLt 



7 . c h A 1 1 e n g i n Q ► . , s I rrip 1 e- 

Uoca.bulary drill: c^ueraqe 3,00 - middle 
I n t e r 3.C t i u e s t or y : a v e r aqe 3.44 = mi dd 1 e 
Hyper speech: range 4.33 - 5.00; auer-s.qe 4.72 = ra tht-r 
c hi a 1 1 e n q i n g 

There is general agreement that hyperspeech lessons are 
rather c hi a 1 1 e n g i n g , but not e r wh e 1 m i n g . This is the 
i ri t e n t , 

3e c t i on 3 : Feature s Use age 

This section was not evaluated because it did not seem 
use-ful and because student responses appeared careless and 
e r r a t i c . We thin k we h bx^ e a go od n o t i on o -f wh at -F e a t u r- e s are 
used . 

Part UIl ~ Personnel Management and Training 

A major part of the project consisted of man aging and 
training the personnel involved in lesson production and 
1 esson presen t a t i on , 

Lesson production; an important factor in the eventual 
wider use of the FLIS system lies in the ability to train 
people in designing and authoring lessons. The details of 
this lie ou t s i de the sc op e of this summary ; h owe v e r s ome 
conclusions can be drawn at this point. To date^ 
a p p r o X i ma t e 1 y 10 people ha v e been trained in the design and 
creation of lessons. All have been able to master the 
system; approximately half after a week or two of training 
and practice, w i t hi o t h e r s r- e q u i r- i ri g se e r a 1 w e e k =• . One or 
two n e V e r r e a 1 1 y bee ame c cmf or table or proficient, Pr i or t o 
this summer, training was done in the following sequence: 
introduction to the computer, operating system essentials, 
lesson design, and finally details of lesson implementation 
using the authoring system. 

This summer a new author was trained in hyperspeech 
lesson creation in less than 3 days total time (spread out 
over two weeks), using a new approach," After a demonstration 
of a hyperspeech lesson and inspection of the hard copy data 
of the lesson, together with a diagrammatic explanation of 
how the lessons work, this new author was put to work on 
successive details of implementation using the authoring 
system. After- having been led step by step through the 
creation of one lesson, and the modification of another j she 
was able to successfully complete an entire lesson on her own 
with minimal help in about a week of half-time work. 
Although my impression is that this individual is of above 
average ability, the superiority of this method seems clear. 
It leads us to believe that people can be trained in a 
limited amount of time to produce hyperspeech lessons quickly 
and independently. Further experience in and development of 
author training techniques would be an important component 
of making this system truly portable. 
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other personnel i n'-..' o 1 'j ed in 1 e = son production were: 
native speakers to transcribe and do rough translations of 
lesson rria t e r i a 1 , n a t i 'j e speakers to record t hi e audio, and 
artists to p r- o du c e suitable pictures tor- e a c hi pi a r t o -f e a c h 
lesson. No serious or systematic problems other than the 
usual ones of coordination and c omm u n i c a t i on w ere 
encoun ter ed . 

Less on presentation : t r ad i t i on a 1 1 an gu a g e 1 abs t yp i c a 1 1 y 
require personnel to manage lab e q u i pme n t an d rtia t e r i a 1 s . At 
least at present, this CAI system is no different, We did 
have a lab assistant present at all tirries when students were 
taking lessons, to locate and load the various disks for each 
lesson, and to provide assistance to students in the event of 
confusion or trouble with the e q u i p rri e n t , The latter f u ri c t i on 
was especially irtiportant since we did not have t i rrie for a 
leisurely introduction and orientation for the students. 

The person hired had considerable experience with micro- 
computers, and had no trouble learning what was required. 
About 2 hours of initial orientation and training we re g i v e n , 
foil owed by occ as i cu"ia 1 br i ef supp 1 emen t ar y c on su 1 t a t i on s . 

It had been hoped that this person could also observe 
studen ts tak i ng 1 essons i n sortie systemat i c fash i on to 
de t-.-rrtji ne problems, styles of using the lessons, etc., but 
t h i s p r o V e d i rrip oss i b 1 e . Request s and q u ^ s t i on s c arrie a t 
u n p' r- e d i c t a b 1 e i ri t e r^,' a 1 s , and pi r- e v» e n t e d this kind ci f 
monitoring. In the future, if any systematic observation of 
students taking lessons is desired, a specific person with no 
other duties at that time ig i 1 ] have to be present. 

Par t UI I I - Prof i c I ency Exam 

In accordance wiith one of the aims of the grant, we 
d e V e 1 opi e d a p r o t o t y p e listening c omi p r e h e n s ion t e ^. t . Tw e n t ;■ 
multiple choice questions r an g i n g in difficult y f r om =• i rrxp 1 e 
(typical greetings) to c orrip lex (level of e du c a t i o rr a 1 reform 
in count r y X > w ere developed. Each q u e s t i on t oo k the form of 
a statement in the foreign language; then 4 possible 
responses to the statement were played. The student was 
then to choose the most appropriate response. The lesson, 
originally developed in E n g 1 i sh , was then translated < i t h 
mi i nor miod i f i c a t i on s all ow i n g for language and culture 
variations) into the 7 Southeast Asian languages taught at 
SEASSI , and every student was asked to take the test for the 
language he or she was studying. Not all actually showed up, 
but approximately 30 students took the test. 

In order to validate the tests, teachers of e a. c h c 1 a. s s 
we re asked to rank their students < wh o t ook the test). 
Detailed results are included in an appendix. A brief 
c om p u t a t i orf of rank correlations for each c 1 as =• foil ow s : 

1, Beg i nn i ng Tha i : 5 students rho 
2 , I n t e r mie d ) a t e /ad v an c e d Th a i : 9 s t u de n t s r Y\ o 
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3 , Burmese: 6 ^. t u d e n t s 

4 . T 5^ g a 1 o g ; 7 students 

5 . L c« : 6 e- t u d e n 1 9- 

6. Khymer: 14 students 

7. ^'i e tnamese class A; 5 students 
S. Uietna.mese cl.=tss B: 5 students 

9. V i e t n3.mese cIass C: 5 students 

10. I n don e s i A n (not yet a'v' i 1 1 e > 

In general, these s^re strong positive correlations, 
indicating that the test agrees with teacher eualuations. The 
relatively low correlation for the beginning Thai class is 
not readily explainable, except that with small samples, one 
or two maj or d i sagr eemen ts wou 1 d c ause a s i gn i -f i c an t drop . 
One or two students getting lucky could also h av e s i gn i f i c an t 
effect. The negative and low is.'alues for 'v'ietnamese classes B 
and C are probably due to the fact that the teachers* for 
these students spoke a different dialect than that used in 
the test. 

Altogether these figures suggest that the prototype 
tests are quite valid, and could be used to quickly judge an 
i n d i V i du a 1 •* •=. listening c omp r e h e n s i on level. For add i t i on a 1 
details, please refer to the appendix, which was prepared by 
the SEASSl language coordinator. 

Par t I X ~ Con elusions and Re c ommie n da t i o n s 

- have students take lessons under better conditions: 
n o t wh en the y a re tire d ari d h o t . 

~ p r o vide s ome e x t r i n s i c m o t i v a t i on grades ) 

- abandon or redes i gn vocabul ary dr i 1 1 1 essons 

- develop more interactive story lessons as opportunity 
and authoring talent are available 

- develop more hyperspeech lessons similar to the last 
one described. Decide upon and standardise the test method. 

- try some hyperspeech lessons for beginning students. 
These would have to be written, not 'found'. 

" m o d i f >' the p r o gr am s o t hat the rria i ri p e e c hi i ^. ri o t 
automatically repeated after each hint is played. 

~ look at frame usage patterns as displayed in lesson 
statistics printout. 

- improve and clarify questions on the CAI Lesson 
Eval ua t i on Form . 

- watch audio quality carefully 

- develop and test training for new lesson authors. 
This really could use a full ID treatment, if resources were 



rho = .VS 

rho = .7? 

rho = .70 

rho = .7/. 

rho = .97 

r h o = - . 40 

rho = .50 



A p p e n d i c e =• 

1 . Samp 1 6- CAI Lesson E'j=i] uat i on Form 

2. Summary of CAI Lesson Evaluation Forms 

3 . P r o t i c i e n c y Test Summ ar v 

4. FLA paper: " Compu ter -Con t ro 1 1 ed FCandom Access Audio 
In the Comprehension Approach to Second Language Learning" 
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Appendix II: Saumple FLIS Pictures (reduced) 
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Appendix III: FLIS Listening Proficiency Exam Experiment 

(Summary published in COTSEAL Bulletin. 1988) 
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Summary of FLIS Listening Proficiency Exam published in 
COTSEAL Bulletin 2.2/3.1 1988. 



tenth week. In addition, [the 1987] SEAS SI used an unofficial proficiency standard 
whereby each supervising teacher, in consultation with his or her colleagues, gave a rating 
similar to the FSI scale of each student's actual performance in the four modalities. Copies 
of the proficiency ratings will be sent on to Hawaii. In instances where a student is 
continuing through several years of SEAS SI, it will prove useful to be able to track his/her 
performance over time and, perhaps much later, to use the same records to study attrition. 

Prototype Listening Comprehension Test 

In a rather interesting experiment just before the close of the 1987 SEAS SI, a pilot 
project was designed to see what potential the NIU Computer-Aided-Instruction (CAI) 
system had for proficiency testing purposes. With funding from a grant from the U.S. 
Department of Eiducation, SEASSI supervising teachers of the major languages were paid a 
modest stipend to assist in design, translation and recording. 

Twenty multiple choice questions ranging in difficulty from simple (e.g., greetings, 
apologies) to complex (e.g., educational reform in country X) were developed. Each 
question took.the form of a statement in the foreign language; then four possible responses 
to the statement were played. The student was then to choose the most appropriate 
response. The lesson, originally developed in Enghsh as a convenient starting point, was 
then translated (with miiior mo(fifications allowing for language and culture variations) into 
seven of the nine languages taught at the SEASSL Every student was asked to take the test 
for the language he or she was studying. Not all actually showed up, but approximately 80 
students took the test on an individually scheduled basis. The text took 20 minutes and 
was scored automatically by the software. 

In order to validate the tests, teachers of each class were asked to rank their students 
who actually took the test in global terms, best to weakest in general language proficiency. 
A brief computation of rank correlations for each class follows: 



1. 


Beginning Thai. 


5 students 


rho = 


.38 


2. 


Intermediate/Advanced Thai 


9 students 


rho = 


.64 


3. 


Burmese 


6 students 


rho = 


.98 


4. 


Tagalog 


7 students 


rho = 


.79 


5. 


Lao 


6 students 


rhO = 


.70 


6. 


Khmer 


14 students 


rhO = 


.76 


7. 


Vietnamese, class A 


5 students 


rho = 


.97 


8. 


Vietnamese* class B 


5 students 


rho = 


.40 


9. 


Vietnamese, class C 


5 students 


rho = 


.50 



10. Indonesian (teacher correlations incomplete) 

In general, these are strong positive correlations, indicating that the test agrees with 
teacher evaluations. The relatively low correlation for the beginning Thai class is not 
readily explainable, except that with small samples, one or two major disagreements would 
cause a significant drop. One or two students getting lucky could also have significant 
effect. The negative and low values for Vietnamese classes B and C are probably due to 
the fact that the teachers for these students spoke a different dialect than that used in the 
test. Altogether these figures suggest that the prototype tests are quite valid and could be 
used to quickly judge an individual's listening comprehension level. Used over time, and 
in conjunction with other tests of proficiency (e.g., a CAI reading proficiency exam), a 
reasonably accurate profile of each student's language competency could be maintained for 
a number of purposes: placement, diagnostics, course design, etc. 

Some of the students who took the CAI listening proficiency test made the imponant 
point that, in retrospect, their course of instruction had not really emphasized 
development of higher-level listening skills being tested by the computer. They 
were enthusiastic in reconimending more focus being placed on such training (such as is 
done at the Defense Language Institute and the Foreign Service Institute). Finally, we note 

1 1 



10^ \ 



o. ■"■ A : ! r • r :_ r ■ ; E r l: c t ! C: n I?- ■ ^ ■ " ^" r ^ ■" 



\ r. I'-. =cc^ 



nstruci: i on : l i sten to so-: 



i. 



^ i t u 8 ^ I or: g i V' •= ~ i ri En g ] • vl-. 



: t 



W ill be io] \ ■rAi.yr: di by 4 s t a t e.Ti t ■=■ : ^ ^« ior \ Q • if Q u H Qt- y.ou 
k n oi.'.i . You i i 1 i ndi Co. i.oh i ch o^'^h* o+ the 4 is n"iO= t apn ropn j .^--rf^ 
response to the Qiven s i t sj t i on by choosirig A, ::• , C, on 



(NOTE TO THE EXAM PREPARER; The choices A-D ^.r e ;n i:he 7a,rQ,^t 
1 a.n yu e.qe . Pie ase c h e.n ge t h e c h o i c e s t o nris.k e t h errj ■= m i t e.b 1 e t o the 
t an ge t 1 5.ri gu age a.ri d c u 1 t u r e i r\ or de r t o 1 n su r- e t e s t y =! ; i d r t >■ - 
The Engl i sh gi'-.-'en here is suggest' v.' e o-F i.-ohat := 9.p p n op " : t e ; rj 
E Ti g 1 i ■=. hi o ri 1 >■■ a. n d m a >■■ n o t t r 5. n e- 1 t e • t o t ri e t r- g e t \ s r: c a g e 
w I t h ou t d { =- 1 :t- t i on o-f t e s t r- su ^ t ■:- or r odu c e n n a t '.j r a ■ 
1 an guaqe . 

For sake o-f consistence/ across languages, please -follow the 
format -for the CORRECT ANSWER <A-D) 



^ kTO^^K \ Vou are -ForPigr: student. You see .=!. woman p rot esse n 

*S(f^^ i '^^^y^^^' P '-J '^^ • ^' u greet he r b >■■ s- a y \ ri g : 



on 



r^lTTTTl'^a'ri^L^J:/^ suggested responses into the target 

i a.nguage \ c h a.age- -ttT?- '- ?i un't-^tv--s^trLUj:J;.u r- e f ,^vJfer/r!=rg1^^ ' ' =• =■ > ' r5rh:-e^th^e rri 
a»^-p-p-ap« r- i a t e to the target language, \-i /ou uj i sh . ) ' ~ 



A. Ho..'s i^^^^o^n-^ B. 
u . HP e you o , 



Hey, mar' C. Good morningj professor, 

■ '--C ''^-^^Jl§:„^or_rjr c t^ res 




f You help B. crippled per=on cross a -street, He thanks you by 
saying, " -'-{x^Xc^'^'^'^o^ •" 'jou respond b>" saying; 

. ( target 1 ar-guage) Y^t\ 'I • ^ r^^^^' y/^ B- '"IJ^ P^' f']'^'" ^ 

I. You are in a bus and you -^tep on the foot o-^^ an eiderly 



] ady beh i nd you . You say to h^- ; 
A. TL correct refeponseA B. 1'^*^ y \| 



You see an e mp t y sea t a t a t ab 1 e n the sc h oo i c af e t e r I a : 



girl of about IS years of age is siting there by herself 



vMOuld like to sit there » so ycAj say 



I C' u 



(.aTL L B. [correct 




ter/i ask-i- you 



.^l}:^. tbt^. 



A wai terJi asks- you what you would 1 ike, 'i'ou say: 



s.sK-s- you what you would 1 : kt- , 
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I n r. t r ;j c r i izi r. ^ 
o.r. c r- c-' D^! ■=l?•=■ 



r 




h rs g I ' -^-h > I n -e- t r- u c t i <:■ r; s 
r e =■ pi o ri =• e i*..' h i c h: b 8 s t c orrs o ] t t ^ 

!< T' Q ^' t 1 r.: il; IJ i<. Q 9 "f Ci 1 1 iZil,'.) S . . ' 



L . Hit 



v 1^ Ti y} w ^ ■ J L K 'J — • 



X ; E c use rri , o ^e* -=^^1 i •= hi u •=. t o p 

A,tj eric r; E-Ti b a ■= >" \ 

"v" ; ^•io , • do^ -.n 

riumbe r- 44 . / 

/TEST \PREPARER: CHANGE THE RE9PCNsEs ,TQ"^liITABLE T.L. SITUATION 
1 / Cor-rectvx::esporiSe is in bold -fac 

A . V ou" re right, sir, B Ex c ^ me . I don *■ t k n o;." . 
C^^ Thank you very much, siry D. oLJ rc- ^'je 1 c orrip 



,1 




■•II,, At 8. c o.Tip a n y o+ -f i c ir 



X: Wha.L rooni is Miss o?Jchi.=«ds ' n.;-' 

Y : i n t h a t ori e , sir-. W-=t. \ k s t r- I gri t e ^.u ; it' s on t h 

Ye s J it " ■=■ t h a. t r oorri . B . I dorr ■■ t n o'>/* . 
"^h-^nk-E Tor aski^^q. D, .Qh"', yes- I see which one. 



ill S. i c. S=-r OC'ITi T 

T e 5. c i"! e r- : 



! -J >_f e I ' 



I wcLi id like /oi-. s t 'J c-? n t s t o h a.n d you r h orriev-jor ^" 
cj n M ci n d a y = 

■!i.'jho ui i '1 t ^ E-bser.: o"- l-lor- c: =^.;- -e ". o Vhe 

tea. '-her : 



IX. 



A. 

B, 
C. 



I can-'t come home, pro+essor. b. C^'.n my friend i.Mr i te i t*:-^ 
Are you kidding? D. May I hand it in on Tuesday, pro-fessor? 



! n a. de n t i s t '* s oi -flee. 

Dentist: What seems to be th.^ probl errr? 
p3.tlent: I ha'..'e h.H.d •=<. tooth^>.che ror three d.=«.-: 

"^h e de n t i s t s the n say s : 

I'll buy 3 small tubes o-F to: thp -h-^ ie . ^ 
Is it a. ta.l se tooth? 
Open your mouth and 1^11 have a. look. 

Sorr >■ , th i s i =.n ' t a den t i st ■ e o-f i ; c-; 
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<r^ (Q ■• • i'lht-ri? ^- ^ / ou going-' 

I ; V I ■■ rri qc i rig to S i n rja.pcir e . 

a: 1/1 h 5. t rjl,>^.-Es tr5. in? 

Pi . F o ur^j^r<r- s o n s b . i h e y d r- e = •£ r r- a ; ri , 

e c on d class D . B >- t e e e r- i n -i; t r ^ \ 



L. _ 

P5.rt C. Eue-vthino is in th e target 1 anguass- .\ \ yAHe^t*^ / 



11 Xl . Ther? is a kriock at the door, ■•-cu op^-n tn* door etig ^.ee a 

v' ' frierid. You say: 



' A. Hex, why are you here? ^ B. He' :Q= ^-^d.y I he-p ycu? 

C . P 1 ease c ome i n an d s i gp. >■ ojj r - arrse . 
D . Th an k s + or t h a door . 



^ ?5 XI!. You rriee. a friend. Her father has Jjjst died, "'"ou sa>- ; 

S'O li^-^A. I'm sorry to hear that.^B. Conora tu 1 ^ t i ons ' 

1 C. I =«.m most gratetui. D. Piea=.e rr-ake yourself at ""crrje . 



y , XIII. You see a vendor ■■■female-' sell ing bananas. You i.-MCL- "■ d ike 

7 I t o b u y om e . "< o u s .=«. y ; 



. Oh, are you sell ing bananas today? 
C . Ar e t h e se ban an as -f or sa ] e ? 



\3 ^ B. How much are these banan3.s?t/ 

D . tJ ho is s e l 1 ing t hi a s e a n a n a =• 



/ XIU. You are in a pol ice =.tatior:. You h^^e lost you^' u^a i ■ t and 

I passport. The po- ice officer asks you ;f you can r erriemb'?" your- 

p- a =■ E- p o r- 1 1^ IT; t" e r • r ■» ' ci -j -iz^ z- ;■• ; 

S^iP" m' J ^ j .j. p t-membe r where I lost it. 

B. Yes J i t s here in my wal let. / 

'*C. Mo. I ii-tn 't remiember the number 

D, I t s none of you r bu s i n e ss ■ 

^ K'v' . You go to a bank to open an account. You ask th'r bank 

'o-f-ficer: 



A. Can I buy this bank? B. Where do I go to open an account? 
C. Where can I ujithdraiAi my m»oney now? D. I m out of mo"-. e/. 
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i( f\ V ..^ 

i-v^i i ch :■ c-L E "• =? c t- E •: ■' . .. " " I I ' " _ .. . ' : ■ 
J / C". M: e 1 >• E. H.V'z to - c-'j -■ '-. .t- : : ' . ••; • r r •: . ~ . -^r; 
"^(i ' -^.to j t i her uncle, ; S r z^^ 1 1 ^ : n > : " ■ - ■: 

her i.g ! t h c u : d-^vn^.E^ 9 i n q - i - r -a- ' r ^ . c- r z !"• ; c- : c -i'm.. ■! " : 

A, So yuu ■=«.r€' .=«. member of th ; r corrujr-t ^.it: ■ 

3 . I c^m ^- o r- r >- t *"!■=«. t rri > -j ri c ■ : =■ c o r- u u t . 

C . I •= y o I J r- 'J n c ' e r- ii? ; ' > • ri p o 1 ^ t 1 c •= /■ 



D. I'm sorry ^^aTI didn't Know you were related. 



XUI I . Your mother ^.sV.-:- ^ou in ir-'jitt- rc=ur bo-=.=. to ■=» hc'-. = -^~ 
wsrming party she is or g.=i.n i z ; rsg , t'.-- tr^M h i rr* : 



\^ /a. Could you pick up some Deer- n^or me? 
^ B, My mother would like you to come to a house-warming party on . 

Sunday, i -f you are not too busy. 
C. My mother s-ays that i -f you don't c :-me to :he h ou se -i-iarm i r:-:. 

party, I should oe t a r.-ei-^^ boss, 
D , I i y o u c onri e t c« ui h ij s- e -i-'j a r m i ri 9 p a r t >- , c: :j •: n m e e t m >• 'n r. = s . 

Z t A colleague or cla-rmate asks yoL' to ccinr;m:5=rir on his r'r..:=er- or; 

the rriost recent election. Yo'j do not 1-5 an t to oe d : = hor-.^s - , bu i: 



t he p> a p e r- i s e r >•■ pi ij or 1 y w r i 1 1 e n . Y o u ■= a y : 

A . It ■'■ s oby i c«ij s t ri a t >c«u k n oi-! n c 1 1-> > n g abou t p o i t i c ■=■ . 

B. Why don't you ask your boy-friend to re=*d it"' It's boririO. 

C. You'ue got some good ideas in your paper j but it needs more 
.re-*wr i ting. 

D . I'll give you some c omme n t s , but it \ ] ] be e p ~ e . 



f tV XIX. There is -h debe'e beti/oeen two evtren'v^ o-^ the nnlitice: 

snectrum on how to ;rr:proi.;p t-.e eoLCst sy = tev; ". v-rjr 

ouril;'-v. Yd J ^re =^ ^::?e--jt^ .-^-;?- ~h • r- : ; - t 



A. I think that !\ie =-ouid a.bo] i sh thv requ ; i"-n-:^n t c-f c c~p u 1 so: >• 
e ducat i o n of S y e a r - s rii i n i m u m . 

B . The M i n i s t e r oi De + e n z-e sh ou 1 d be a 1 1 Qi.vie d t o de s t r 8. 1 \ 
M i 1 1 age school e and ^-;o^. e ch i 1 dren into pr ou i nc ; a 1 cen t ers . 
C . I n e 'v' e r e n t t o •= c h o o 1 and I e e r, o r p .=1 o n i.-i h y e d u c a t ^ o n s 
necessary tor others. 

0. I 'firmly believe that we could improve education by 
increasing teachers'" salaries and -funding more educational 
research • 
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<?-r.ri^r:'- p - -lip he ^ h ^ Sec jf' " ' '■■=■• . ^■'■■^ =: * : v e - ■ 

t ] ; s you IhEi. ^ .r-ciij ri^ed npec.T' p ■? r -n • r r , or. r ■- ^-.r*? *:' it o-i?-: t ~ " = , 

A. I nt-e-d to -z-t* ^ th9S^ doc ume f; "i: "lod-?. y a.ric: ■: ol*-^-"9"" 

B. I ^.m ■=<. u;^ = 1 -known uir i t^r- xPiC rhou' :: '«c-!: - a- e p r-rr. ■ = • - r- - - .- r-. 
anyone' . 

C. In that case, can you tel] me how I proceed to get 
permi ss i on? 

D. In my co;jritr>-, i-m»? don-'t need p e rri! . 5:.= ; on to ust- the =^ z-^' 

Hr-r h 1 e =- . 



'■Pl'='=tse k:eep th:-= ex^^.m con^ ■ den ^ i -e.. ' . .ocv r .jp or thrcj : : 

5>^>^s:y , i -f you do not intend to u^e i". unco t i •=. -n -e- t de r; t = ■= 

h5«nds-, it is no longer 1 id ^. t j ' ■ and 'Aie '"i ' M ^^e^-d t 

constr-ijct a r?eui one at ne!.-'- coat in t = rne a^ic ' abi-'~- . - - - Tnr". e . 
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student Answer Sheet: To be used in trial test in classroom or 
in Language Lab. PS 



Name : 

Language : Level : 

I n ■=. t r- u c t ; or- = i i •=■ t e r. t r- t r, p i n ■= t i. i r t . r-r, •= , = r = : ^ t ; nr^ op d .^j, r.c- 

C'T'd tnen choose the mosi; apc-r- opr- i 3. te r e = p o^- = £. , Cr or- [= . 

i . a b c 'iH 

2. c- b c d 

3 . H b id 

4. a b c d 

5. a b c d 

6. a b c d 

7. a b c d 
3. a b c d 
9. a b c d 
16 . a b c d 
1 1 . a b - d 



13. a b c d 

14. a b c d 
1 5 . a b c d 
1 6 . a b c d 
1 7 . a b c d 

15. a b c d 
19, a b c d 
2e b c d 
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plehSE '^■rv this ROLi;?~ :-Rh^f- ?!s"^ mtth :-.r.ME ::.r - 

QRALLV IN CLASS USING -ENCIL Ar^D PAPE^" . STUDENTS ^.l^-J-E - -■ S. S ■ , 



) ! ! ■■| ' Ci u iTi ^. y c h a. ri g e t •? =. u g q e •=■ t ^ d r' €• s p> o ri e s t c m c r- e s u i - .=« -• ' e c n e 
i r« your particular- "i ^.r. gu-^ge . Do not sti-ik to -^r-- ex^-.rt 
t r?.n=. 1 a.t I on o-f tht' or-'oiri?: Eng^ i ^.h , !t /OU thini' st i.'i-' produce 
i fj V' a "i id test r w i t ■= . 

J oh n P , Har trri r^.n n 

Ld.n Quage D i r c t or , SE^SS I Sc /S? 



SEASSl 1S87 

Rough Drafts Prof > c i *:?nc>- E>;?.rri ot Corrimu r. : c a t ■ t CoriO c- : c«? 

C: om f ■ r- ^? hi e ri -=. i o n - E: a •=■ e d L cirn p u t e r- e^i ■=. ■= : •= t e d 

I n t p odu c t i on : Th i e is a 2Ci -Item t-- x .E-m o-f i i ■= t e r. - n g c " ^ h e e ; c 
ij •= i ri q 1 i 'J e sp e c h r e r o r- d e d o n a n I n e- t a v o g . e . E -e- c - ^ * ^- »=■ 2 ? 
j t arris i n c 1 u da s sorri'E i n f or ma t i on or d ? ■=< 1 og ? Ir. ou t a r-.^ aj": : r.- o-^ u ' 
s : t J E* t i on ■: O'-' ^ r ■ - q : ■• - q^ ■ . : :~r- : ' : ■ ^ e - •- :• ■■ ■: ■ : 

studant ha.r«rs the situation described o^- th-E ■=ho'* t d'E^lci.. he or 
■=.i'i a ill hear ^ .i h o e c e ^. , i e^ b e 1 e d A , 3 , . I* . > : c ^- " : ■_ I 

r 9 sp o n se '5 i n d i c e« t e d by typing one o-f the -r o u ^- : a ^ t e s . "^r v 
student- s per- t ormance i j ; 1 1 be conrtp'^ted E':^ t o*;a t i c e ' " ny 
cor7;riuter and a p-irt-:::.t o^ the p e r o---; e n - - :•- : ^ -f-- : - - i • : 5 . 
■ • ■ oe e'/ a i ] at • e . 

No t e : Th i •=■ e x am : e or: "i > on e ci E-e e r- =i. ■ i n = t r- ume n t s t h ?. t c e.n be 
u se d t o p r obe a s t u de n t s p r o-f i c i e n c >■ . Mor e o'.- e r , it test s on 1 >■ 
Q n e d I m e ri s ion: 1 i s c e n I n n r rim p r- e hi e ri •= i Ci n , i.- i h; : c hi w o u ] d i ri c ^ u d e 
aure) memory. It should be used with other tests: -f^ e>c e- t o--^ ac e 
interviews, in-class te=-ts o-f compr-eheris i on o-f listeninG an.: 
reading, and tests o-f production in speaking and wr i t i ri g . 

Tht e de y e 1 op me n t C"-t t hi i s p r- o t i c J e n c a :■■ arri i s a r e E-e =*.r' c n p r- c=..! e 1 1 
•funded by the U .8 . Deo t . of Educat i on . ; 
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I Introduction 



Most computer-aided instruction (CAI) for foreign language is altogether 
silent, depending entirely on mute text and occasional graphics to instruct 
? and aid the learner. A few systems incorporate a tape recorder, which either 
forces a rigid, invariant lesson design or requires the learner to endure delays 
. of up to half a minute before hearing a desired speech segment. CAI systems 
* that incorporate interactive video are a promising alternative, but they will 
..be expensive, probably will not be customizable, and commercial versions 
will not be available for the less commonly taught languages due to high 
production costs. 

.The Northern Illinois University Foreign Language Instruction Station 
4 . (FLIS) is based on a computer-controlled random- access audio device called 
;^the Instavox and consists of a complete lesson authoring and presentation 
rvSystem that can record and play back high-quality audio segments (speeches) 
i^in conjunction with computer-displayed text and graphics. Any audio segment 
f Specified by the lesson author or chosen by the learner can be accessed 
^- and played with no delay at any time during the lesson. Speech samples 
f^of any desired content can be provided by the lesson author and can be 
...palled up by the student at will as the lesson progresses. These audio segments 
I .might include hints, translations, cultural notes, interactive feedback messages 
; or slower and more distinct re-recording of the original natural language 
segment. 
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With a moderate investment in equipment, training, and time, any indi^ 
or institution can create, modify, and use FLIS-mediated interactive 'ac 
lessons for any language (or any subject where audio via Instavox is a ii^ 
added dimension in the presentation of instruction). A preliminary v( 
of the FLIS system was completed in mid- 1985; presently some ten ai 
have been trained to write materials on the FLIS system, only two of ^ 
have had any significant previous computer experience. While the rate ^ 
with which individuals master the FLIS authoring protocols, in most ca 
a week or two of training and practice are sufficient; moreover, the author 
process is entirely menu-driven. Thus, no knowledge of computer prog 
ming is necessary. 

This report offers first a brief description of the equipment and sof 
that composes FLIS, including specific features seen by both lesson auti 
and students, and then characterizes two innovative lesson types ("interactiv 
story" and "Hyper-speech") designed to improve Ustening-comprehensi^ 
skills. The report concludes with some comments on evaluation efforts t 
date. 



The FLIS System 
Equipment 



The current FLIS system consists of four workstations, each equipped with 
an Apple He computer and an Instavox (a direct- access audio device). The 
four workstations are connected by a Corvus network to a Corvus Haidl 
Disk Drive, which holds the FLIS computer programs and student records. | 
The data needed for an individual lesson are stored on two standard floppy ^ 
diskettes, one containing pictures for the lesson, and the other holding | 
information about sequencing of lesson material, speeches to play back, ; 
specific feedback for given responses, etc. 

The audio component of the lessons is provided by the Instavox, which ^ 
accomplishes what would be impossible with a tape recorder— instant access 
and replay of any speech at any time during the lesson with a delay of | 
less than half a second. The Instavox is similar to the conventional floppy- 
disk drive of a computer but employs a Mylar disk 15 inches in diameter 
that holds up to thirty minutes of recorded (as opposed to machine-generated) 
human speech in which all the nuances, tones, and rhythms of language 
are reproduced faithfully. 

At each workstation used in the teaching of Indonesian and Thai at the 
NIU Southeast Asia Center, the Instavox is controlled by the computer using 
lesson data that indicates corresponding software programmed to store the 
exact location of the beginning and ending of each speech segment. Access 
to audio segments is random in the sense that requests to play back speech 
samples may be made in random order and may differ from one use to 
another in the lesson. In this fashion, the lesson author may choose to have 
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speech samples played or replayed in an order that depends on the particular 
responses of the learner, or the learner may be given some degree of control 
in the progress of the lesson, perhaps ebcting to hear optional hints or 
translations that are made available, or choosing to repeat or skip certain 
optional sections of the lesson at will. In either case, the Instdvox audio 
de^'/ice is able to access these random locations with virtually no delay. 
Immediate access and playback result in a significant saving of learner time 
and a greater efficiency in the learning task when compared to other systems, 
such as the random-access tape recorder, which demonstrate delays of up 
to ten seconds to access a given speech. 



The Authoring System 

Creating a lesson for the FLIS system is a two-part task. The lesson must 
be planned carefully, bearing in mind the capabilities of the system as well 
as its limitations. The general lesson type is determined first (four have 
been developed so far: drill, tutorial, interactive story, and "hyper-speech''; 
the lattfci two are described below), then individual "frames** are designed 
as needed. Each frame constitutes a single computer- student interaction; 
for example, a single frame may consist of presentation of material, followed 
by a question, the student's answer, and the computer s response to that 
answer. Frames may include multiple-choice or short-answer questions or 
may offer a choice of what happens next (for example, a branch at a new 
section, the playing of a hint), or may require no action at all from the 
learner. Along with this detailed planning of the scope and content of the 
lesson, the author may choose to provide more or less detailed remediation 
or feedback to incorrect answers. In addition, a script of speeches and pictures 
[ to be used at each point of the lesson must be prepared. 
\ Once an author has produced one lesson of a given type, subsequent lessons 
\ are more easily created, since the underlying framework will be familiar, 
^ although the content and details may be quite different. 
I After the lesson is planned and scripted, the actual production is made 
I easy by the FLIS Authoring System. The system is entirely menu-driven 
I and requires only that the author know what should happen in the le;,£on 
\' at each point. All lesson data can be imprinted for later reference, and all 
\' data can be changed at a later time, if desired. This circumstance means 
I that the lesson itself can be modified in light of difficulties or shortcomings 
I discovered in the course of formative evaluation. Typical information for 
\ a frame might include the following: the speech, text, and picture to use 
? in the "presentat''^.-" part of the frame, the specification of possible correct 
L and wrong answers, and the response the computer should give for each 
% answer. The computer's response can be (1) none (in which case the lesson 
^proceeds to the next frame), (2) a message displayed on the screen.. (3) 
an audio message, or (4) a branch to a set of remedial or speech feedback 
? frame(s) for that particular answer. 
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Especially noteworthy is the Instavcx component of the authoring $yi 
which controls the recording of speeches. This, too, is menu- driven,^' 
allows either live recording of individuals or the copying of segments 
audio tape or cassette to the Instavox disk. The author simply taps? 
computer's space bar to iriitiate the recording (or copying) process and 
taps it once again to stop. Audio segments may be recorded automatic 
one after another, or the recorded segment can be replayed instantly, bj 
tapping a key; if the recording proves unsatisfactory (too loud or soft, 
lively enough, background noise, the last syllable cut off prematurely, etcig 
the speech samples can be re-recorded and validated instantly. In additio^ 
a given speech segment can be re-recorded at a later time or additional 
speeches added to the lesson. The extreme ease of creating and editing 
the audio materials for a lesson contrasts sharply with other video or Gp| 
ROM devices. 

O 

Lessons 

Each FLIS lesson may be included in a number of **courses'* of lessons 
and made available to students. Each student is registered in a specific section 
of a given course. A choice of up to thirty lessons is displayed for each 
course with an indication of which lessons have been tried previously by 
the particular student. Once a choice of lesson is made, the student is 
responsible for obtaining and inserting the two floppy disks and the Instavox 
disk into their respective machines, and the lesson begins. At the end of 
the lesson a summary of the student's performance is displayed on the screen 
and written to a permanent file on disk, and the student is returned to the 
course menu to retake the lesson, choose another, or quit. 

A primary challenge for authors who design lessons for the FLIS system 
is to go beyond traditional drill or tutorial lessons to other types that make 
use of the unique capabilities of random-access audio. While traditional 
lesson types are useful to a degree, the Instavox capacity for random- access 
audio invites teachers to create lessons that can take fuller advantage of 
the computer's capacity for interactive instruction. 

Two innovative lesson types for FLIS developed at Northern Illinois 
University that appear to have great promise are "interactive story" and 
**hyper-speech.*' Both are explained more fully in Henry, Hartmann, and 
Henry (See Reference). 



The Interactive Story 

Interactive story lessons were in spired by the structure of the series of "Choose 
Your Own Adventure" stories written for children. In these books, at the 
end of each page the reader is invited to choose what will happen next 
and, depending on the choice, is instructed to turn to a particular page. 
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In short, the reader "branches" through a series of alternatives, whose election 
at each point determines the subsequent story line and its consequences. 
The story, then, has many variations, and can be read many times, each 
with a different outcome. 

The extension of this concept to FLIS-mediated lessons is straightforward; 
the computer poses choices to the student at various points in the interactive 
story lesson and, based on the student's response, continues with the chosen 
part of the plot. For example, in one such FLIS lesson, the student takes 
the part of an Indonesian child sent to the market to buy fruit for a guest. 
At various points in the lesson, the student must make choices for the child 
in accordance with what is heard: which kind of fruit to buy, whether to 
buy at the offered price or to bargain; whether to accept a lower offer or 
to walk away (in hopes of a still lower offer); whether to buy some fruit 
that is not quite sweet enough, etc. At each point the student sees a picture 
of the situation and overhears the voices of the child and the sellers. Not 
all frames are "choice" frames; in some the narrative is interrupted and 
the student is asked a question (Does the child have enough money to accept 
the seller's offer?); in others the student may simply be asked to listen and 
understand perhaps with the aid of optional hints or translations. 

Students are encouraged to undertake lessons of this kind several times, 
each time making different choices. In this way, a considerable amount 
of repetition of lexical items and syntactic patterns is experienced, but the 
repetition is cloaked in sufficient variety to prevent boredom. The interest 
inherent in discovering the unknown consequences of the choices appears 
to be a powerful motivating factor if the "plot" is cleverly constructed. An 
additional attraction of interactive story lessons of this type is the ease with 
which the consequences of actions in different cultural settings can be 
illustrated unobtrusively. For example, in Western culture, bargaining is rare, 
and the strategy of walking away from the seller to elicit a lower offer 
is not widely practiced. In the FLIS-mediated interactive story lesson, this 
situation (and others of cultural contrast) may be presented and the results 
of various actions demonstrated in a vicarious but dynamic fashion. 

The Hyper -Speech 

In its pure form, the hyper-speech lesson gives the student a set of speech 
segments to understand and, for each such segment, a set of instantly available 
aids to comprehension. Aids may include the following: 

1. Instant repetition of the speech segment 

2. A set of audio hints (lexical, grammatical, or cultural notes) 
? 3. A picture illustrating the situation 

V 4. A slower, clearer re-recording of the original speech segment 

5. The text of the speech segment displayed on the screen 

6. Written notes about the speech displayed on the screen 
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7, A translation of the speech segment, spoken or displayed 

8. A glossary of unusual words in the lesson , 
Typically, a learner is required to access some of the less complete aid^ 
before full text display and translation will appear, but other than that 
restriction (which need not be followed by the lesson author), students are* 
free to attack the audio passage as they please. Evaluators engaged in on-; 
site observation of students working with hyper-speech lessons have noted 
that some learners prefer to repeat the passage many times before calling^ 
up any of the aids; others turn almost at once to the re-recording and tbea-. 
l?ack to the original. Still others exhaust all available hints before continuing; 
Students exhibit considerable perseverance in trying to comprehend as long? 
as the listening passages are not too far beyond their ability; perhaps more' 
important, students report that the lessons are enjoyable and seera very* 
worthwhile. This positive evaluation can be ascribed, perhaps, to these reasons: I 
mature students apparently like the fact that they are in control of their 
own learning, and can set the pace of the lesson and the details of their 
interaction with it. In addition, the nature of the materials and the way 
they are presented offer the user a clear sense of accomplishment in 
understanding samples of authentic foreign language instead of the controlled 
and contrived speech typical of classroom language. Finally, the materials 
reflect the full range of environments where speech is natural, purposeful, 
and functional (taped conversations, commercial recordings of comedians, 
news broadcasts from the radio, songs, plays, etc.) and where the content 
is clear, relevant, and interesting. Hyper- speech lessons are probably not 
suitable for beginning students, but seem clearly suitable for students at 
the intermediate level or above. The concept may be applicable for beginning 
learners, however, with carefully selected (or crafted) materials and their 
equally careful workup for use on FLIS with special attention to the hints 
and other aids appropriate for beginning students. 

Evaluation 

Lessons that incorporate direct-access audio presented by an interactive 
computer system appear to be effective and highly motivating to students 
learning foreign languages. The equipment and time necessary to produce 
these lessons, while far from negligible, are moderate compared to any other 
presently available medium that can claim usable random-access audio 
capability; moreover, they are not beyond the means of many secondary 
schools or institutions of higher education. The Northern Illinois Universit>' 
Foreign Language Instruction Stations, together with the lesson types that 
have already been developed, constitute a viable means to explore further 
the possibilities of random-access audio in all levels of second-language 
learning. 
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